WO/2006/100664, of the present inventor, discloses systems, methods and computer readable code for providing a single player and/or a multi-player gaming environment. Each user is provided with a client device having an input module and display screen. An application server array receives data indicative of game commands from each client device. Residing in the application server array is a game engine which maintains a virtual gaming universe including one or more of user-controllable gaming characters. The gaming engine is operative to associate each user-controlled gaming character with game commands received from a respective client device of the plurality of devices. At the application server array, one or more streams of videos are rendered. In some embodiments, each video stream represents a view associated with a different game character. Each video stream is sent over a communications link to a respective client device. In some embodiments, the communications link is a switched network, such as a packet switched network of a circuit switched network.
It is noted that the WO/2006/100664 is incorporated into the present disclosure by reference in its entirety. Citation of the aforementioned reference (or any reference) does not constitute an admission that the cited reference is prior art.
The present disclosure relates to improved systems and methods for providing gaming services and to improved techniques for encoding and/or transmitting and/or decoding video. It is noted that although the presently disclosed teachings are applicable to the systems described in WO/2006/100664, this should not be construed as a limitation, and it is noted that the presently disclosed teachings are applicable in a variety of systems including those not described in WO/2006/100664.
It is noted that when implementing a system where (i) a gaming engine for multi-player game resides on the server-side; and (ii) video content descriptive of the virtual gaming universe is rendered on the server side and streamed to client devices, it may be desirable to address one or more of the flowing issues: (i) game latency; (ii) game fairness issues in a system where server-client game latency may be present; (iii) video streaming, for example, via a wide-area network; (iv) rendering video appropriate for different client devices each having different device parameters; (v) general issues related to improving the gaming experience; (vi) jitter related to variations in latency.
Game Latency
There is an ongoing need for reducing server-client latency in systems where video is rendered at the server side and/or a view of a virtual world is generated at a server side.
In the example of FIG. 1, a game engine generates a snapshot S(t0) (either rendered or unrendered) of a virtual gaming universe for a given time (for example, t0) and sends this snapshot to a client device (for example, a mobile telephone or a personal computer or any other device having video display functionality). Unfortunately, due to client latency, this snapshot is not immediately displayed to the user—rather, the snapshot generated at ‘real’ time tr0 is only displayed at a later time tr1 due to ‘game latency.’ In particular, it is noted that ‘game latency’ may come from one or more of a number of sources, including but not limited to (i) game-engine latency associated with computing the updated state(s) of the virtual world or universe; (ii) rendering latency associated with rendering a particular view of the updated virtual world or universe (iii) encoding latency associated with encoding the rendered video (iv) network latency associated with streaming the encoded video from the server-side to a given client device.
In the example of FIG. 1, the latency is the difference between tr1 and tr0.
Thus, a greater game latency may, in some situations, provide for a less responsive gaming experience. Even worse, in some situations, the game latency may adversely influence ‘game fairness’ as will be explained in the next paragraph with reference to a particular example.
In this example, two gaming characters are fighting each other to the death. Each gaming character is controlled by a respective user client device which sends commands—whichever client device can react to a situation within the gaming universe faster will enjoy an unfair advantage.
In this example, the network latency may vary among different client devices, and some client devices may experience very different latency than other client devices. In this example, the game server is located in Tel Aviv Israel and client devices are located in Haifa, Israel and Boston, Mass. According to this example, an updated description of a virtual world (for example, a next ‘frame’) arrives at the Haifa-based client device hundreds of milliseconds before the same description arrives at the Boston-based device. In this example, the Haifa-based client is thus allowed to react to the updated situation in the virtual universe at an earlier time, giving the user of the Haifa-based client device an unfair advantage. Thus there is an ongoing need for minimizing game latency associated with multi-player gaming systems (for example, provided in a wide-area network) and for handling fairness issues associated with differential client latency.
There is an ongoing need to for apparatus and techniques which minimize the latency and thus provide a more realistic and responsive gaming experience.
There is an ongoing need to for apparatus and techniques which allow for handling of game latency in a manner that provides more ‘game fairness.’
Multi-User Games Provided Using Different Types of Client Devices
One feature disclosed in certain embodiments of WO/2006/100664 is rendering of views of a virtual gaming universe on the ‘server side.’
To date, most multi-player and many single-play games are marketed as ‘multi-platform’ games where the user can play the game from a plurality of platforms. In one example, this allows the game to be played, for example, using different types of PDAs or cellphones. For example, some users among a user population operate more “sophisticated” client devices (PCs, ‘advanced’ cellphones, PDAs). These users expect a “richer” gaming experience that utilizes the graphics and/or sound capability of the client device—for example, a game that displays more and richer landscape features (or more features of certain game objects, or displayers a game object or landscape feature with greater contrast), that utilizes a higher device screen refresh rate for a “smoother” experience, etc. Other users operate more “primitive” client devices and require game software that is less resources-intense (e.g. graphics resources, display resources). In another example, a single user may want to play a given game using his PDA when “on the road” and using his desktop computer when in the “office.” This user expects a richer gaming experience using the desktop computer.
Unfortunately, development of multi-platform multi-player or single-player games is extremely expensive. The game must be “tweaked” for each platform. Thus, in one example, certain objects, which might be important for conducting the game, are not displayed on a less-sophisticated display device, and developers must develop, debug and maintain a version of the game software that does not display these objects, or that displays the objects with less contrast.
It would be highly desirable to have systems and methods that facilitate porting of single player and/or multiplayer games between multiple display platforms and/or multiple devices without loosing critical graphical details in the various views.
The present inventor has discovered that certain features of the system and method in WO/2006/100664 may be adapted for providing a technique or providing multi-user
Techniques for Handling Encoded Video
The MPEG-2 Draft Standard provides temporal redundancy reduction through the use of various predictive and interpolative tools. In many systems, three types of frames or pictures are used: “I” Intrapictures, “P” Predicted Pictures, and “B” Bidirectional Interpolated Pictures.
The “I” Intrapictures provide moderate compression, and are access points for random access, e.g., in the case of video tapes or CD ROMS. As a matter of convenience, one “I” Intrapicture is provided approximately every half second. The “I” Intrapicture only gets information from itself. It does not receive information from a “P” Predicted Picture or “B” Bidirectional Interpolated Picture. Scene cuts preferably occur at “I” Intrapictures.
“P” Predicted Pictures are coded with respect to a previous picture. “P” Predicted Pictures are used as the reference for future pictures, both “P” and “B” pictures.
“B” Bidirectional Coded pictures have the highest degree of compression. They require both a past picture and a future picture for reconstruction. “B” bidirectional pictures are never used as a reference.
Motion compensation goes to the redundancy between pictures. The formation of “P” Predicted Pictures from “I” Intrapictures and of “B” Bidirectional Coded Pictures from a pair of past and future pictures is a key feature of the MPEG-2 Draft Standard technique.
Thus, typically these pictures are sent to a display device as a sequence, wherein after sending a single I picture, a number of B or P pictures may be sent. Typically, the I pictures are sent periodically to provide a “refresh.”
The present inventor has noticed that there are some situations where the aforementioned scheme for encoding and decoding video content may not be optimal. Some situations relate to certain implementations of certain systems described in WO/2006/100664, though this is not to be construed as a limitation to any currently-disclosed technique, and is not to be construed as a limitation to any technique disclosed in WO/2006/100664.
There is an ongoing need for improved techniques for encoding and/or decoding electronic video content.